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ABSTRACT 

The  goal  of  this  paper  is  to  compare  the  accuracy  of  two  approximate  confidence  interval 
estimators  for  the  Bernoulli  parameter  p.  The  approximate  confidence  intervals  are  based 
on  the  normal  and  Poisson  approximations  to  the  binomial  distribution.  Charts  are  given  to 
indicate  which  approximation  is  appropriate  for  certa.a  sample  sizes  and  point  estimators. 


'This  research  was  supported  by  the  National  Aeronautics  and  .Space  Administration  under  NASA  C'on- 
tract  No.  NASl-19480  while  the  authors  were  in  residence  at  the  Institute  for  Computer  Applications  in 
Science  and  Engineering  (ICASE),  NASA  Langley  Research  Center,  Hampton,  VA  23681-0001. 


1 


1  Introduction 


There  is  conflicting  advice  concerning  the  sample  size  necessary  to  use  the  normal 
approximation  to  the  binomial  distribution.  For  example,  a  sampling  of  textbooks 
recommend  that  the  normal  distribution  be  used  to  approximate  the  binomial  distri¬ 
bution  when: 

•  np  and  n(l  —  p)  are  both  greater  than  5  (see  [1],  page  211,  [5],  page  245,  [7], 
page  304,  [9],  page  148,  [16],  page  497,  [17],  page  161) 

lies  in  the  interval  (0, 1)  (see  [15],  page  242,  [12],  page  299) 

•  np(l  —  p)  >  10  (see  [13],  page  171) 

•  np(l  —  p)  >  9  (see  [1],  page  158). 

Many  other  textbook  authors  give  no  specific  advice  concerning  when  the  normal 
approximation  should  be  used.  To  complicate  matters  further,  most  of  this  advice 
concerns  using  these  approximations  to  compute  probabilities.  Whether  these  same 
rules  of  thumb  apply  to  confidence  intervals  is  seldom  addressed.  The  Poisson  ap¬ 
proximation,  while  less  popular  than  the  normal  approximation  to  the  binomial,  is 
useful  for  large  values  of  n  and  small  values  of  p.  The  same  sampling  of  textbooks 
recommend  that  the  Poisson  distribution  be  used  to  approximate  the  binomial  dis¬ 
tribution  when  n  >  20  and  p  <  0.05  or  n  >  100  and  np  <  10  (see  [8],  page  177,  [5], 
page  204). 

Let  vYi ,  ^2, . . . ,  X„  be  iid  Bernoulli  random  variables  with  unknown  parameter 
p  and  let  Y  =  3,  binomial  random  variable  with  parameters  n  and  p. 

The  maximum  likelihood  estimator  for  p  is  p  =  ^,  which  is  unbiased  and  consistent. 
The  interest  here  is  in  confidence  interval  estimators  for  p.  In  particular,  we  want 
to  compare  the  approximate  confidence  interval  estimators  based  on  the  normal  and 
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Poisson  approximations  to  the  binomial  distribution.  Determining  a  confidence  in¬ 
terval  for  p  when  the  sample  size  is  large  using  approximate  methods  is  often  needed 
in  simulations  with  a  large  number  of  replications  and  in  polling. 

Computing  probabilities  using  the  normal  and  Poisson  approximations  is  not  con¬ 
sidered  here  since  work  has  been  done  on  this  problem.  Ling  [11]  suggests  using 
a  relationship  between  the  cumulative  distribution  functions  of  the  binomial  and  F 
distributions  to  compute  binomial  probabilities.  Ghosh  [6]  compares  two  confidence 
intervals  for  the  Bernoulli  parameter  based  on  the  normal  approximation  to  the  bi¬ 
nomial  distribution.  Schader  and  Schmid  [14]  compare  the  maximum  absolute  error 
in  computing  the  cumulative  distribution  function  for  the  binomial  distribution  us¬ 
ing  the  normal  approximation  with  a  continuity  correction.  They  consider  the  two 
rules  for  determining  whether  the  approximation  should  be  used:  np  and  n(l  —  p) 
are  both  greater  than  5,  and  np(l  —  p)  >  9.  Their  conclusion  is  that  the  relationship 
between  the  maximum  absolute  error  and  p  is  approximately  linear  when  considering 
the  smallest  possible  sample  sizes  to  satisfy  the  rules. 

Concerning  work  done  on  confidence  intervals  for  p,  Blyth  [2]  has  compared  five 
approximate  one-sided  confidence  intervals  for  p  based  on  the  normal  distribution. 
In  addition,  he  uses  the  F  distribution  to  reduce  the  amount  of  time  necessary  to 
compute  an  exact  confidence  interval.  Using  an  arcsin  transformation  to  improve  the 
confidence  limits  is  considered  by  Chen  [4]. 

2  Confidence  Interval  Estimators  for  p 

Two-sided  confidence  interval  estimators  for  p  can  be  determined  with  the  aid  of 
numerical  methods.  One-sided  confidence  interval  estimators  are  analogous.  Let 
Pl  <  P  <  Pt/  be  an  “exact”  (see  [2])  confidence  interval  for  p.  For  y  =  1, 2, . . . ,  n  —  1, 


2 


the  lower  limit  pi  satisfies 


k=y  \  / 

where  y  is  the  observed  value  of  the  random  variable  Y  and  a  is  the  nominal  coverage 
of  the  confidence  interval  (see,  for  example,  [10],  page  279).  For  y  =  1,2, —  1, 
the  upper  limit  pu  satisfies 

This  confidence  interval  requires  numerical  methods  to  determine  pi  and  pu  and 
takes  longer  to  calculate  as  n  increases.  This  interval  will  be  used  as  a  basis  to 
check  the  approximate  bounds  reviewed  later  in  this  section.  A  figure  showing  the 
coverage  probabilities  for  bounds  of  this  type  is  shown  in  Blyth  [2].  Following  a 
derivation  similar  to  his,  a  faster  way  to  determine  the  lower  and  upper  limits  can 
be  determined.  Let  VFi,  IV2, . . . ,  be  iid  U(0,  1)  random  variables.  Let  Y  be  the 
number  of  the  kF,’s  that  are  less  than  p.  Hence  Y  is  binomial  with  parameters  n  and 
p.  Using  a  result  from  page  233  of  Casella  and  Berger  [3],  the  order  statistic  W  =  VF(y) 
has  the  beta  distribution  with  parameters  y  and  n  —  y  +  1.  Since  the  events  Y  >  y 
and  W  <  p  are  equivalent,  P[Y  >  y]  (which  is  necessary  for  determining  pi)  can  be 
calculated  by 


P{Y>y)  =  P{W<p) 

=  - - -  r  -w)^-^dw. 

r(y)r(n-y  +  1)  Jo 

Using  the  substitution  t  =  and  simplifying  yields 


P{Y  >  y) 


r(n  +  l)  +  ^  , 

T{y)T{n-y  +  iy  y  ’  Jo  (-:J±i+<)n+i 

pr/r  ^  (n-y  +  l)p. 

c  l^2y,2(n-y+l)  <  J’ 
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Since  this  probability  is  equal  to  q:/2  for  a  two-sided  confidence  interval, 


^  («-y  +  l)PL 


or 


In  a  similar  fashion, 


PL 


1 

1  I  "-v-H 

V^J».2(n-K+l).l-a/2 


PU 


1 


1  + 


_ Jira _ 

(V+l)^2(»+I),2(n-|,),o/2 


The  next  paragraph  discusses  numerical  issues  associated  with  determining  these 
bounds. 

The  Mathematica  (see  [18])  code  for  solving  the  binomial  equations  numerically 


is 


pi  *  FindRoot[ 

Sum [Binomial [n,  k]  ♦  p  *  k  ♦  (1  -  p)  “  (n  -  k) ,  {k,  y,  n}]  alpha/2, 
{p,  y  /  n}  ] 


pu  =  FindRootC 

Sum [Binomial [n,  k]  *  p  '  k  *  (1  -  p)  “  (n  -  k) ,  {k,  0,  y}]  ==  alpha/2, 
{p,  y  /  n}  ] 


for  a  given  n,  y  and  a.  This  code  works  well  for  small  and  moderate  sized  values 
of  n.  Some  numerical  instability  occurred  for  larger  values  of  n,  so  the  well  known 
relationship  (Larsen  and  Marx  [10],  page  101)  between  the  successive  values  of  the 
probability  mass  function  f{x)  of  the  binomial  distribution 


/w  =  /(- 1) 


X  =  1 , 2, . . . ,  n 


x(l  -p) 

was  used  to  calculate  the  binomial  cumulative  distribution  function.  The  Mathemat¬ 
ica  code  for  determining  pc  and  pu  using  the  F  distribution  is 
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fcrit  »  Quantile [FRatioDistribution[2  *  y,  2  *  (n  -  y  +  1)]  ,  alpha/2] 
pl»l/  (l+(n-y+l)/(y*  fcrit)  ) 


fcrit  =  Quantile [FRatioDistribution[2  ♦  (y  +  1) ,  2  *  (n  -  y)] ,  1  -  alpha/2] 
pu  *  1  /  (  1  +  (n  -  y)  /  (  (y  +  1)  *  fcrit)  ) 


This  method  is  significantly  faster  than  the  approach  using  the  binomial  distribution, 
but  encounters  diflBculty  with  determining  the  F  ratio  quantiles  for  some  combinations 
of  n  and  y. 


The  first  approximate  confidence  interval  is  based  on  the  normal  approximation 
to  the  binomial.  The  random  variable  7^""^  is  asymptotically  standard  normal. 

v/np(l-p)  ^  ^  ^ 

Thus  an  approximate  confidence  interval  for  p  is 


where  2a/2  is  the  1  —  a/2  fractile  of  the  standard  normal  distribution.  This  approxi¬ 
mation  works  best  when  P  =  \  (e.g.,  political  polls).  It  allows  confidence  limits  that 
fall  outside  of  the  interval  [0,  1].  One  should  also  be  careful  when  T  =  0  or  T  =  n 
since  the  confidence  interval  will  have  a  width  of  0. 

The  second  approximate  confidence  interval  is  based  on  the  Poisson  approximation 
to  the  binomial  (see,  for  example,  Trivedi  [16],  page  498).  This  confidence  interval 
does  not  appear  as  often  in  textbooks  as  the  first  approximate  confidence  interval. 
The  random  variable  Y  is  cisymptotically  Poisson  with  parameter  np.  Therefore,  the 
exact  lower  bound  pi  satisfying 


i;  ( t )  pio  -  plt-'‘ = «/2 

can  be  approximated  with  a  Poisson  lower  limit  ppi  which  satisfies 

^  (npPL)^  e-"”" 

2. - li - = 

k=y 
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(«Ppl) 


p-nppL 


- =  «/2. 

fc=0  *• 

The  left-hand  side  of  this  equation  is  the  cumulative  distribution  function  for  an 
Erlang  random  variable  with  parameters  nppi  and  y  (denoted  by  ^nppt.y)  evaluated 
at  one.  Consequently, 

<  1)  =  a/2 

Since  2nppi,Enppi^,y  is  equivalent  to  a  random  variable  with  2y  degrees  of  freedom, 


this  reduces  to 


P[xly  <  2nppL]  =  a/2 


1  2 

PPL  -  ^  X2y,l-a/2- 


By  a  similar  line  of  reasoning,  the  upper  limit  based  on  the  Poisson  approximation 
to  the  binomial  distribution  is 


PPU  —  ^  X2(y-H),o/2- 

This  approximation  works  best  when  p  is  small  (e.g.,  reliability  applications  where 
the  probability  of  failure  p  is  small). 

3  Comparison  of  the  Approximate  Methods 

There  are  a  multitude  of  different  ways  to  compare  the  approximate  confidence  inter¬ 
vals  with  the  exact  values.  We  have  decided  to  compute  the  error  of  an  approximate 
two-sided  confidence  interval  as  the  maximum  error 

max{|pz,  -pi|,|p[/  -pu\} 

where  pi  and  pu  are  the  approximate  lower  and  upper  bounds,  respectively.  This 
error  is  computed  for  all  combinations  of  n  and  p.  Since  the  definition  of  “success” 
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on  each  Bernoulli  trial  is  arbitrary,  we  only  consider  the  range  0  <  p  <  |.  Figures  1, 
2  and  3  have  mirror  images  for  the  range  i  <P<  1. 

Figure  1  contains  a  plot  of  n  versus  p  for  n  =  2, 4, . . . ,  100  and  considers  the  range 
0  <  39  <  I  for  a  maximum  error  of  0.01.  Thus  if  the  actual  error  for  a  particular  (n,p) 
pair  is  greater  that  0.01,  the  point  lands  in  the  “Do  not  approximate”  region.  If  one 
of  the  two  approximations  yields  an  error  of  less  than  0.01,  then  the  pair  belongs  to 
either  the  “Normal  approximation”  or  “Poisson  approximation”  regions,  depending 
on  which  yields  a  smaller  error.  Not  surprisingly,  the  normal  approximation  performs 
better  when  the  point  estimate  is  closer  to  i  and  the  Poisson  approximation  performs 
better  when  the  point  estimate  is  closer  to  0.  Both  approximations  perform  better 
as  n  increases.  In  order  to  avoid  any  spurious  discontinuities  in  the  regions,  the 
calculations  were  made  for  even  values  of  n.  The  edges  of  the  region  are  not  smooth 
because  of  the  discrete  natures  of  n  and  p.  The  boundary  of  the  approximation 
regions  are  those  (n,j5)  pairs  where  the  error  is  less  than  0.01.  If  the  horizontal  axis 
were  extended,  the  normal  and  Poisson  regions  would  meet  at  approximately  n  —  150. 
Mathematica  [18]  was  used  for  the  comparisons  because  of  its  ability  to  hold  variables 
to  arbitrary  precision. 

If  the  maximum  error  is  relaxed  to  0.04,  then  there  are  more  cases  where  the 
approximations  perform  adequately.  Figure  2  is  analogous  to  Figure  1  but  considers 
an  error  of  0.04.  This  figure  also  contains  the  rules  of  thumb  associated  with  the 
normal  and  Poisson  approximations  to  the  binomial  distribution.  In  particular, 

•  the  rule  labeled  “Rl”  is  a  plot  of  p  =  5/n  on  the  range  [10, 100]  corresponding 
to  the  normal  approximation  rule  np  >  5  and  n(l  —  p)  >  5 

•  the  rule  labeled  “R2”  is  a  plot  of  p  =  on  the  range  [4, 100]  corresponding 

to  the  normal  approximation  rule  p  ±  falling  in  the  interval  (0, 1) 

•  the  rule  labeled  “R3”  is  a  plot  of  p  =  |  on  the  range  [40, 100] 
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corresponding  to  the  normal  approximation  rule  np(l  —  p)  >  10 

•  the  rule  labeled  “R4”  is  a  plot  of  p  —  ^  on  the  range  [36,100] 

corresponding  to  the  normal  approximation  rule  np{l  —  p)  >  9 

•  the  rule  labeled  “R5”  is  a  plot  of  n  >  20  and  p  <  0.05  or  n  >  100  and  up  <  10 
corresponding  to  the  guideline  for  using  the  Poisson  approximation. 

The  n,  p  combinations  falling  above  the  dotted  curves  for  rules  Rl,  R2,  R3,  and  R4 
correspond  to  those  that  would  be  used  if  the  rules  of  thumb  were  followed.  Clearly, 
rules  R3  and  R4  are  significantly  more  conservative  than  Rl  and  R2. 

Figure  3  is  a  continuation  of  Figure  2  for  sample  sizes  larger  than  n  =  100.  Note 
that  the  vertical  axis  has  been  modified  and  the  horizontal  axis  is  logarithmic.  The 
curve  in  the  figure  represents  the  largest  value  of  p  where  the  Poisson  approximation 
to  the  binomial  is  superior  to  the  normal  approximation  to  the  binomial.  Since  this 
relationship  is  linear,  a  rather  unwieldy  rule  of  thumb  for  n  between  100  and  10,000 
is:  use  the  normal  approximation  over  the  Poisson  approximation  if  p  > 

4  Conclusions 

Although  there  are  a  number  of  different  variations  of  the  calculations  that  have 
been  conducted  here  (e.g.,  one-sided  confidence  intervals,  different  significance  levels, 
diflferent  definitions  of  error),  there  are  three  general  conclusions: 

•  The  traditional  advice  from  most  textbooks  of  using  the  normal  and  Poisson  ap¬ 
proximations  to  the  binomial  for  the  purpose  of  computing  confidence  intervals 
for  p  should  be  tempered  with  a  statement  such  as:  “the  Poisson  approximation 
should  be  used  when  n  >  20  and  p  <  0.05  if  the  analyst  can  tolerate  an  error 
that  may  be  as  large  as  0.04”  (see  Figure  2). 
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•  For  sample  sizes  larger  than  150,  the  absolute  error  of  either  upper  and  lower 
confidence  limit  is  less  than  0.01  if  the  appropriate  approximation  technique 
is  used.  Figure  3  should  be  consulted  for  specific  guidance  as  to  whether  the 
binomial  or  Poisson  approximation  is  appropriate. 

•  Introductory  probability  and  statistics  textbooks  targeting  statistics  and  math¬ 
ematics  majors  would  benefit  from  including  tlie  use  of  the  F  distribution  to 
find  Pi  and  pu.  Also,  more  of  these  texts  should  include  the  use  of  the  Poisson 
approximation  to  the  binomial  distribution  for  determining  interval  estimates 
for  p.  These  confidf  .ice  limits  only  require  a  table  look-up  associated  with  the 
chi-square  distribution  and  are  very  accurate  for  large  n  and  small  p. 
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Figure  2:  Approximation  methods  for  confidence  limits  (maximum  error: 
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